1234 



AN INEQUALITY FOR RELATIVE 
ENTROPY AND LOGARITHMIC SOBOLEV 
INEQUALITIES IN EUCLIDEAN SPACES 



Katalin Marton 
Alfred Renyi Institute of Mathematics, Hungarian Academy of Sciences 

June 18, 2012 



Abstract. For a class of density functions q(x) on R n we prove an inequality be- 
tween relative entropy and the weighted sum of conditional relative entropies of the 
following form: 

D(p\\q) < 

n 

Con S t.Y / Pi-D( Pl (-\Y 1 ,...,Y i _ 1 ,Y z+1 ,...,Y n )\\Q l (-\Y 1 ,...,Y i _ i ,Y l+1 ,...,Y n )) 

i=l 

for any density function p(x) on R n , where Pi(-\yi, ■ ■ ■ , Ui—1, Vi+l, ■ ■ ■ ,Vn) and 
Qi(-\xi, . . . , Xi—i, Xi+i, . . . , x n ) denote the local specifications of p resp. q, and pi 
is the logarithmic Sobolev constant of Qi(-\x\, . . . ,Xi—i,Xi+i, . . . ,x n ). Thereby we 
derive a logarithmic Sobolev inequality for a weighted Gibbs sampler governed by the 
local specifications of q. Moreover, the above inequality implies a classical logarith- 
mic Sobolev inequality for q, as defined for Gaussian distribution by L. Gross. This 
strengthens a result by F. Otto and M. Reznikoff. The proof is based on ideas devel- 
oped by F. Otto and C. Villani in their paper on the connection between Talagrand's 
transportation-cost inequality and logarithmic Sobolev inequality. 
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1. Introduction. 

The motivation for this paper was to prove logarithmic Sobolev inequalities on 
product spaces, under possibly general conditions. 

First we define some basic concepts: 

Definition. For probability measures p and q on W 71 (m > 1 integer), we denote 
by D(p\\q) the relative entropy of p with respect to q: 

D(p\\q)= [ log f^dp(u) if p«q, (1.1) 

and oo otherwise. We always have in mind probability measures absolutely con- 
tinuous with respect to the Lebesgue measure, and denote by the same letter their 
density functions. If p and q are density functions on M. m then 

D(p\\q)= [ p( u )log^\du if p«q, (1.2) 

and oo otherwise. If Z and U are random variables with values in W 71 and dis- 
tributed according to p = C(Z) resp. q = C(U), then we shall also use the notation 
D(Z\\U) for the relative entropy D{p\\q). 

Definition. For measures p and q on IR m , the Fisher information of p with respect 
to q is defined as 

i{ph) = [ 

if log(p(u) / q(u)) is smooth. 

Definition. The distribution q on M m satisfies a logarithmic Sobolev inequality 
with constant p if 

£>(p||g)<~/(p||g) 
for all density functions p on IR m with \og(p(u) / q(u)) smooth. 

A logarithmic Sobolev inequality for a probability measure q is equivalent to 
the hypercontractivity of the diffusion semigroup associated with q. The prototype 
is Gross' logarithmic Sobolev inequality for Gaussian measure which is associated 
to the Ornstein-Uhlenbeck semigroup [1] , [2] . Another use of logarithmic Sobolev 
inequalities is to derive transportation cost inequalities (a tool to prove measure 
concentration), c.f. F. Otto, C. Villani [3]. The logarithmic Sobolev inequality 
for the stationary distribution of a spin system is equivalent to the property called 
"exponential decay of correlation" ; for this concept we refer to Bodineau and Helffer 
[4] and Helffer [5]. 



Vlog 



p{u) 



q(u) 



p(du), 



(1.3) 
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In Euclidean spaces of dimension greater than 1, no simple characterization is 
available for the measures q satisfying a logarithmic Sobolev inequality with some 
positive constant. A well-known sufficient condition was given by Bakry and Emery 
[6]: A density function q(x) = exp(— V(x)) on IR m satisfies a logarithmic Sobolev 
inequality provided V is uniformly strictly convex. Another useful result is Holley 
and Stroock's perturbation lemma [7] which asserts that if q and q are density 
functions on IR m , such that the ratio q(x)/q(x) is bounded both from above and 
below, then q and q either both satisfy a logarithmic Sobolev inequality, or neither 
of them does. 

For measures on Euclidean spaces with non-compact support, it has been a chal- 
lenging task to derive logarithmic Sobolev inequalities from logarithmic Sobolev 
inequalities for the local specifications. (The local specifications of the measure q = 
C(Xi, . . . , X m ) on IR m are the conditional densities Qi(-\xi, . . . , Xi-i, Xi+i, . . . , x m ) = 
C(Xi\Xi = xi,...,Xi-i = Xi-i,X i+ i = x i+ i,...,X m = x m ).) Let q be a density 
function on a Euclidean space, and assume that the local specifications of q satisfy 
logarithmic Sobolev inequalities with constants pi. It has been clear for a long time 
that a reasonable approach to prove a logarithmic Sobolev inequality for q is to 
assume that the mixed partial derivatives of V(x) = —logq(x) are not too large 
relative to the numbers p^. This line was followed by B. Zegarlinski [8] and, follow- 
ing in his footsteps, G. Royer [9], Theorem 5.2.1). Their results were improved by 
F. Otto and M. Reznikoff [10]. The present paper follows this line, too. The con- 
ditions of Otto and Reznikoff 's main theorem helped to find the proper conditions 
for the results in the present paper, however, our approach is entirely different from 
their's. We shall discuss Otto and Reznikoff 's theorem at the end of Section 2. 

2. Statement of the results 

Let R N denote the A^-dimensional Euclidean space equipped with the Euclidean 
distance and the Borel a-algebra. 

Let us fix a density function 

q(x) = exp(-V(x)), xeR N . 

We shall use the following 
Notation: 

• q : a fixed density function on M N ; 

• X = (Xi,X 2 , • • • , X N ) : random sequence in R N , £>{X) = q; 

• p : another density function on M. N ; 



• Y = (Y"i, Y 2 , . . . , Y N ) : random sequence in M. N , £(Y) = p; 
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(Ik, k = 1, 2, . . . , n): a partition of [1, N], \Ik\ = n^; 
for x G R", = { Xi : i G 4}, = {^i : i £ h); 



and the corresponding segments of X; 

and the corresponding segments of Y; 



p(fc) a £(y(k)j ) p (fc) (-|j/ (fe) ) = = yW). 



We consider as the product of Euclidean spaces IR^-* of dimension rik- 

Definition. The conditional distributions Q^ k ' (-\x^ k ') and p^ k \-\x^) are called 
the local specifications of q resp. p. 

To formulate the main results of this paper, we also need the concept of (average) 
conditional relative entropy, together with some more notation: 

Definition. If we are given a probability measure n = C(S) on R e (i > 1 integer), 
and conditional distributions fi(-\s) = C(Z\S = s), v(-\s) = C(U\S = s) on W 71 
then consider the average relative entropy 



D(p\\q)<--Y,Pk-D(p^ k \-\Y^)\\Q^(-\Y^)) for all p on R N , (2.1) 



' k=i 

for some positive constants pk, 1 < k < n, and p. I.e., we want to bound D(p\\q) by a 
weighted sum of the "single phase" conditional entropies D (p^ (-lY^^WQ^^lY^)). 
A bound of type (2.1) holds only for a restricted class of probability measures q, 
and we want a sufficient condition for (2.1). Since relative entropy measures in a 
way how different probability measures are, inequality (2.1) allows us to conclude 
to closeness of p and q from the closeness of their local specifications. Moreover, 
an inequality of type (2.1) ensures that upper bounds for the "single phase" rela- 
tive entropies D(p^ (-\y^)\\Q^ (-\y^ k ')) that hold uniformly in y^ k \ yield a bound 
for D(p\\q). This is a way to get logarithmic Sobolev inequalities for measures on 
product spaces. 

To get inequality (2.1), we make three assumptions explained below. Recall that 
(Ik, k = 1, 2, . . . , n) is a partition of [1, N]. 




For a fixed measure q on IR , 



we want to derive an inequality of the form 



n 



P 
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Assumption 1. Assume that Q( k > (-\x^ k ') satisfies a logarithmic Sobolev inequality 
with constant p k for all x G R N and k G [1, n]. 

Consider the Hessian of V(x) = — logq(x) : i.e., the matrix (Vij(x)). N y 
where we denote by Vij(x) the second partial derivatives of V(x). 

Assumption 2. Assume that, for each k G [l,n], the matrix (Vij(x))ij^i k is 
bounded from below by some (possibly negative) constant times the identity. 



To formulate Assumption 3, we introduce the following 

Notation. Under Assumption 1, and for sequences G R N fixed, we denote by 
A(x, £) the matrix with elements 



for i G I k ,j G I £ ,k ^ £, 



Aij(x^)=0 if i and j belong to the same set I k - 

Moreover, for sequences x, £ G M. N and < p < minp/j, we denote by A p (x^) the 
matrix with elements 



KM 



for iel k ,jel e ,k^£, 



A P j(x^) = if i and j belong to the same set I k . 
(Thus A(x,Z) = A°(x,Z).) 



Remark. Unless the matrix A p (x,^) is constant in x, it is not symmetric, since in 
the definition of A p -{x,^) (i G I k ,j G we use and not 

Assumption 3. We assume that 



and that p is such that 



8u P ||A(x,o|| = i-<y<i, 



sup||i4"(x,0||<l. 



(2.2) 
(2.3) 



Conditions (2.2) and (2.3) shall be used in the following form: For all x, £,u,v G 



fc,£e[i,n],fc^ iei k ,jeii 



<(l-8) 



>J fe=l 



(fc)|2 . 



\ £=1 



(/)|2 



(2.4) 
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and 

k,£e[l,n],kjt£ i€l k ,jel e 



£>-p)>W| 2 , (2.5) 
respectively. 

Theorem 1. If Assumptions 1-3 hold then 

D(p\\q)< -■J2pk-DU k \^)\\Q^ k \^)) (2.6) 
P fc=i ^ ' 

for any probability measure p on M. N . 

Theorem 2. Under Assumptions 1-3, q satisfies a logarithmic Sobolev inequality 
with constant p. 

Theorem 2 follows from Theorem 1, using Assumption 1 and the fact that by 
the definition of the operator V 

n 

I(p\\q) = J2El{p( k \.\Y^) || Q( fc )(-|y^)). 
k=i 



$> fc -pH«<*>| 2 



k=i 



\ 



The statement of Theorem 2 was proved by F. Otto and M. Reznikoff [10], 
under a condition similar to, but stronger than, Assumption 3. We discuss Otto 
and Reznikoff 's theorem at the end of this section. 

Next we formulate a logarithmic Sobolev inequality for a discrete time Markov 
process governed by the local specifications (-\y^). 

Definition of weighted Gibbs sampler. 

Given a partition k = 1, 2, . . . , n) of [1, AT], and local specifiations Q^ k \-\y^) , 
the weighted Gibbs sampler V with weights {ji^ k \k = 1,2, ...,n) is the Markov 
operator on the probability measures p (on M. N ) defined by 

n 

r = J2 T*r*, r fc (z|y) = %<*>,*<*>) • q<*> (*<*> \y^). 
k=l 

(Here 5 denotes Kronecker's S.) 
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Corollary to Theorem 1. 

If Assumptions 1-3 hold then for the weighted Gibbs sampler V with weights 

(p k /R,k=l,2,...,n), J R = ^/9 fc , 

k 

we have 

D(p\\q) < (D(p\\q) - D(pT\\q)y (2.7) 

Thus 

1-£J -D{p\\q). 



(2.7) follows from Theorem 1 by the inequality 

D{pT\\ q )<^j^p k D{pT k \\q) 

k=i 

(a consequence of the convexity of relative entropy) and the identity 

^(p|k)-^(pr*||«) = ^(p ( *H-l^ ( * ) )IIQ ( * ) (H^ (fc) ))- 

(2.7) can be considered as a logarithmic Sobolev inequality for the Gibbs sampler 
r. Indeed, for the Markov process defined by T, it bounds relative entropy (from 
the stationary distribution) by the decrease of relative entropy along the Markov 
process. 

Next we formulate a transportation-cost inequality that follows from Theorem 
2, using the Otto-Villani theorem (Theorem 1 in [3]). We need the following defi- 
nitions: 



Definition. The quadratic Wasserstein distance between the probability measures 
r and s on R m is defined as 

W(r, S ) = inf[ J B 7r |e-r/| 2 ] 1/2 , 

where £ and rj are random variables with laws r resp. s, |£ — 77 1 denotes Euclidean 
distance, and infimum is taken over all distributions it = £(£, rf) with marginals r 
and s. 



8 



AN INEQUALITY FOR RELATIVE ENTROPY 



Definition. A probability measure s on IR m satisfies a transportation-cost inequal- 
ity with constant p if 

W 2 (r,s) < - -D(r\\s) 
P 

for all probability measures r on M m . 



Transportation-cost inequalities are useful in proving measure concentration in- 
equalities. A transportation-cost inequality for the case when q is Gaussian, was 
proved by Talagrand [11]. Otto and Villani generalized Talagrand's inequality as 
follows: 

Otto and Villani's theorem for Euclidean spaces. [3], [12] 

If a density function on M, m satisfies a logarithmic Sobolev inequality then it satisfies 

a transportation- cost inequality with the same constant. 

By Otto and Villani's theorem, Theorem 2 implies the following 
Theorem 3. 

If Assumptions 1-3 hold then q satisfies a transportation- cost inequality with con- 
stant p. 

In [13], corrected in [14], the statement of Theorem 3, for equal p^s, was proved 
modulo an absolute constant factor. 

Now we compare Theorem 2 with the result of [10]. 

In [10] the statement of Theorem 2 is proved under the following condition in 
place of (2.3): 

For k, £ e [l,n], k ^ £, and x G M. N , consider the following minors of the Hessian 
ofV(x): 



ieh ,j eh 



and set 

K k} £ = sup\\(K k ,i(x 

X 

Then consider the n x n matrix 

K = (Kk,e) k£e[ln]k ^ r 
(K has 0's in the main diagonal.) Otto and Reznikoff use the assumption that 

K< A({p fc -p}), (2.3') 
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where A({pfc — p}) denotes the n x n diagonal matrix with elements p k — p. With 
the notation 



Vpk - p • y/Pe ~ P 

(2.3') can be written in the form 

K' p < Id, 

where Id is the nx n identity matrix. Since K' p is symmetric, this means that the 
largest eigenvalue of K' p is < 1. The elements of K' p are non-negative, thus, by 
Perron's theorem, the largest eigenvalue of K' p equals || K' p ||. I.e., in [10] it is 
actually assumed that 

|| K' p ||< 1, (2.3") 
which is clearly stronger than (2.3). 

Remark. 

If q is Gaussian then the Hessian of V(x) does not depend on x. Otto and 
Reznikoff's result is tight for Gaussian distributions with attractive interactions. 
(For = R; attractivity means that Vij < for i ^ j.) For q Gaussian and 
R( k ) = ]R, Theorem 2 can be formulated as follows: If || A ||< 1 then q satisfies 
a logarithmic Sobolev inequality with constant p, where p is the largest number 
satisfying 

\\ AP ||= !• (2-8) 

Thus Theorem 2 is tight for those Gaussian distributions q for which || A p || (for the 
p defined by (2.8)) is given by the absolute value of the smallest negative eigenvalue 
(and not the largest positive one). 

Example. 

Assumption 3 is practically impossible to check, except when the mixed partial 
derivatives of V (x) are constants. Otherwise we probably cannot do better than use 
Otto and Reznikoff's theorem. However, if the mixed partial derivatives of V(x) are 
all constants then Theorem 2 may give a better result. Indeed, let V(x) = — log q(x) 
be of the form 

n 

v ( x ) = ^2<Pk(x) + a k ,£-x k -xt, 

k=l k,£e[l,n],k^£ 

where for each k and fixed x k , the single phase density Ck(xk) -exp(— 4>k(xk, Xk)), as 
a function of x k , satisfies a logarithmic Sobolev inequality with a common constant 
p. Theorem 2 guaranties a positive logarithmic Sobolev constant if the matrix with 
elements 

ak/ outside the main diagonal, and otherwise 



K' p = 
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has norm < p. On the other hand, Otto and Reznikoff's theorem guaranties a 
positive logarithmic Sobolev constant if the matrix with elements 

|ofe,£| outside the main diagonal, and otherwise 

has norm < p. To see a concrete example when the first condition holds, but 
the second does not, consider the infinite dimensional Toeplitz matrix B = (bk,e) 
defined by 

6jfc,fc+i = 1, 6fc,fc+2 = -1, h,£ = for £>k, £ (£ {k + l,k + 2}, 
and bk,e = From the theory of Toeplitz matrices (c.f. [15]) we know that 

B ||= 2 • max|cosx — cos(2x)| = ^, 

while for the matrix abs(B) consisting of the absolute values of bk,i, we get 

|| abs(B) ||=2- maxjcosx + cos(2x)| = 4. 

Denote by B m and abs{B rn ) the matrices consisting of the first m rows and columns 
of B resp. abs(B); clearly ||-B m || < 9/4 and lim m _ 5 . 00 ||a6s(S m )|| = 4. Therefore, if 
we take A = B m , and if the functions <f>k(x) in the definition of V(x) are such that 
the single phase densities Ck(xk) • exp(— <j>k(xk, Xk)) satisfy a logarithmic Sobolev 
inequality with a common constant > | then Theorem 2 guaranties a positive 
logarithmic Sobolev constant for q = exp(— V). However, we cannot get this from 
Otto and Reznikoff's theorem. 
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3. Proof of Theorem 1. 

Our approach to prove Theorem 1 is based on the interpolation between the prob- 
ability measures p and q realized by the solution of the Fokker-Planck equation 

d tPt (y) = Ap t (y) + V-(pt(y)-VV(y)), Mv) = p(v)- (3- 1 ) 
With the notation 

h = p/q and h t = p t /q, 
the Fokker-Planck equation (3.1) can be rewritten as follows: 

dtht = Lht = Aht-WhfWV, h (y) = h(y). (3.2) 

We have 

E p logh = D(p\\q) and E pt log h t = D(p t \ \q). 

Our argument heavily draws on the ideas developed in the paper by F. Otto and 
C. Villani [3]. To be able to use the tools of [3], we need the limit relation 

lim D(p t \\q) = 0. (3.3) 

t— too 

To this end we prove a logarithmic Sobolev inequality for q with a much smaller 
constant than claimed in Theorem 2. (It is disturbing that this weak preliminary 
result requires a very lengthy proof.) 

Auxiliary Theorem. If Assumptions 1-3 hold then q satisfies a logarithmic Sobolev 
inequality with a constant C = C(R, p-min, where R = Ylk=i Pk, Pmin = minfc pk 
and 5 = 1- sup^ ||A(a;,f)||- 

For the proof of Theorem 1 we also need the following simple lemma ( c. f. (32) 
in [3]). 

Approximation Lemma. In the proof of Theorem 1 we can restrict ourselves to 
the case when V(x) = — \ogq(x) G C°° , and h(x) = p(x)/q(x) is of the form 

h(x) = (1 — s) • g(x) + e, where 

g G C°° is a compactly supported density function (with respect to q), and e > 0. 

The proofs of the Auxiliary Theorem and the Approximation Lemma are post- 
poned to Section 4, although they are used in the proof of Theorem 1 in this section. 

We need some more 
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Notation. 

Let 

Y t = (Y t , 1 ,Y t , 2 ,...,Y ttN ) 

denote a random sequence with C(Y t ) = pt, where p t is the solution of the Fokker- 
Planck equation (3.1). In accordance with the notation at the beginning of Section 
2, we write 

Y t W = {Y t}l : i G 4}, Y t {h) = {Y t ^ : i £ I k ). 

Further, we set 

P f = P ?)(.|#)) = £(yW|?« = #>). 

By the Approximation Lemma we may assume that V G C°° . Then the domain 
of the operator L in (3.2) can be defined so as to contain the class T>q of those 
functions h in C°° that are bounded, and whose partial derivatives of any order, 
multiplied by the partial derivatives of V of any order, are bounded. The class V 
is dense in L 2 (g) and stable under L. 

Again by the Approximation Lemma we can assume that the function ho = h in 
(3.2) belongs to V . As explained in [3], this implies that h t is uniformly bounded 
from below and from above, and that, for t fixed, |V/it| 2 is bounded. (Here we use 
the fact that, by Assumptions 2 and 3, the Hessian of V{x) is bounded from below 
by a (possibly negative) constant times the identity.- In [3] assumption (32) of that 
paper is used which is implied by the assumption ho = h G V q.) 

Consequently, as explained in [3], under condition ho = h G £>o, the Fokker- 
Planck equation (3.2) defines a semigroup of diffeomorphisms 

$ t : R N ^ R N , < t < oo, (3.4) 

satisfying 

^t(y) = -Vlog^($ t (y)), (3.5) 

and 

Pt§ s =pt +S - (3.6) 

(3.6) means that pt+ s is the image of pt under the map $ s . Since C(Y t ) = pt, we 
can think of the random sequences Y t as functions of Y = Y : 

y t = $ t (y) = $ t (y ). 

Let us introduce the function 

n 

x(y) = $> fc [log%) - log y G R N , 

k=l 
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where 



h {k) {y {k) ) = CrS = / %)Q (fc W fc) )ly (fc) )- 



g< fc >(y<*>) U 



<0 



(The integration domain is IR nfe ; the superscript (k) indicates that integration is 
with respect to the variable .) We have 

^ P x = J2p^ D (p {k) (-\ Y{k) ) \\Q ik) (-\Y {k) ))- 

k=l ^ ' 
Thus the statement of Theorem 1 is equivalent to 

p ■ E p log h < E P x- 



It is well known (and a proof can be found in [3]) that 



^D(pt\\q) = -I(pt\\q) = -E pt |Vlog/i t | 2 



Thus, by (3.3), 

POO 

D(p\\q) = D(p\\q) - Urn D(p t \\q) = / E Pt | V log h t \ 2 dt. (3.7) 
We introduce, analogously to the definition of x, the functions 

n 

Xt(y) =J> [log h t (y) - log h { t k) (y(*>)] , 
k=i 

where 

h ( t k) (y {k) )=P { t k) (y {k) )/q {k) (y {k) ). 

We have 

E PtX * = Ep fc - J D(p? ) (-ln ( * ) ) II^W*))- 
k=i ^ ' 

In particular, E Pt Xt > 0. 

Using (3.7) and the fact that E pt x t > 0, for the proof of Theorem 1 it is enough 
to prove the following two propositions: 



Proposition 1. 



E pX -KmE Pt Xt = J E Pt |v X t • Vlog^jdt. (3.8) 
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Proposition 2. 



E p AVxfV\ogh t }> p-E Pt \V log h t 



Proof of Proposition 1. 
For all y G 1^ we have 



f°° d 

x(y) - t iim x*($t(y)) =-y Q^(xt(My)))dt- 



Therefore, by Fubini's theorem, 



E p X (n- lim Xt(*t(Y)) 

1 t— >oo 7 



J™ K{§- t (xt(<f>t(Y)))\\dt. (3.9) 



Denoting by dot derivation with respect to t, and using (3.5): 
d 



dt 



X*(*t(y)) =Xt(^(y))-Vxt(^(y))-Vlog^(^(y)). (3.10) 



Further, 

n 

ft(x*(*)) = £>*• ft(iogM^)) - ft (logfi? (*<*>) 



A:=l 



Z e 



(3.11) 



To calculate ft (log (z^)), we need the following 
Lemma. 

The solution ht of the Fokker-Planck equation (3.2) satisfies 

\\dtht\\h 2 ( q ) < ||^o||L 2 (g) < °°- 



(3.12) 



Proof. 

The operator L is defined on a dense subset V of L 2 (g). Moreover, L is symmetric 
and negative definite on Vq. Indeed, by partial integration we have 



(Lf,g) L2(q) = [ (Af - W • Vf)-gdq = - [ V/ • Vgdq. 

Jr n Jr n 



LOGARITHMIC SOBOLEV INEQUALITIES 



15 



It follows that for A > 



((\I-L)fj) u(q) >\ 



2 

M<?)' 



i.e., 



(XI -L)" 1 ||< 



1 

A 



Thus by the Hille-Yosida theorem (c.f. [16]), there exists a contraction semigroup 
(Pt : t > 0) on L2(g) whose generator is L: 

d t Pth = LP t h for h G V , and ||P t || < 1. 

For h G Vq, the solution of (3.2) can be written as h t = Ptho, and since P t L = LP t , 
we have 

d t h t = d t P t h = LP t h = P t Lh , 



which implies (3.12). 



□ 



By the above Lemma, dtht G hi(q), so we can differentiate under the integral 
sign in the next formula: 



djlog hl k \zW)) =d t [ ht(z)QW(d 
\ J Jrw 

J R{k) dt(h t (z))Q^(dzW\zW) 



z ( k )\z^ 



) 



h ( t k) (z( k )) 



r\ -i , , v h t (z) 

d t log h t (z) ■ (k) 



Q< fc >(<fe<*>|z<*>). 



(3.13) 



By the definition of the function h t , 



M^QiV (cfoW | f (*)) = p W (cfoW | f (*)) . 



/f } (*(*)) 
Thus (3.13) implies 

a (log***' (*<*>) 

= / dtloght(z)pl k) (dzW\zW) =E Pt {d t logh t \z^}, (3.14) 

where z^ in the condition of the expectation is a shorthand for = z~( k \ 

Substituting (3.14) into (3.11) we get 

n 

9t{xt(z)) = ^log^(z)-Ep t {^log^|^} 
fc=i L 
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It follows that ^ Pt Xt = which, together with (3.10), yields 

E p ^(xt (*t (X )) ) = -E pt | V X t • V log ^ } . 
Substituting this into (3.9) we get (3.8). □ 



Proof of Proposition 2. 

We prove Proposition 2 for t = 0; for t > the proof is the same. For a function 

g : R N !->■ R set 

V ( *ta=Wi):ie/ fc ). 



We need the following 

Proposition 3. 

For k,£ & [l,n], i, we have 

V (fc) logfeW (y w ) = E P {V« log^|j/W} 



(3.15) 



where tt^' (d^\ dr]^'\y^') is an arbitrary coupling of the conditional measures 
V {'\y ) and Q^(-|y^). (i.e., ix^{d^\ dr]^'\y^) is a conditional density on 
with marginals p^> (-|y^) and Q^'(-\y^).) 



x 



Proof of Proposition 3. 

Since |V/i| is bounded (and \Vht\ is also bounded for t fixed), we have 

V (*>£W(y0))= /" V«f/ l (^,^))-QW(^|^))V^ ) - (3-16) 



Further, 



V ^ ^ ^ ) V | RW exp(-y(yW,r 7 W))^W 
= -V<*)y(yW,eW).QW(eW|yW) 

+ QW(eW|yW) • / V( fc V(yW,^))QW(^|yW) 



V( fe )F(yW,r/W)-V^V(yW,e W ) 
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It follows that 



vW(MyW,eW).gWftW|yW)) 



V< fc >fc(yW 



= QW(eW|yW)) 



Substituting (3.17) into (3.16): 
= / V<*)fc(yW,eW)QW(deW|yW) 

+ / My w ,£ w )Q w (de ( % w )- / V<*V(yW,»7W)QW(d»/W|yW) 

V<*)fc(yW,eW)QW(deW|yW) + ^(yW). / V<*V(yW r,^)Q^(d V ^\y^) 

) ./rot 
/»(y(0, e W).v(*)y(yW, e (0)QW (de (0|y(0). 



Dividing both sides by h^\y^')\ 

V(*)log^(yW) = / V^logMyW £W) • ^^Q W (^^|y^) 
./rW hW{yW) 

+ [ V^V(y( i \ V ^W i \d V W\y^) 
Jrw 



Since 



and 



^||^<?«'»(<i««'»|y«'»)= P < (^< |y<'»), 



/ vWlog/ 1 (s(",«('>y«)(d«(')|s(')) = E 1) {v(*)log/ 1 |s(')}, 

JRM 

(3.15) follows. □ 



18 



AN INEQUALITY FOR RELATIVE ENTROPY 



Now we are ready to prove Proposition 2. By Proposition 3 we have 



V (fc) x(2/) = E 



Pi 



V (fc) log%) - V (fc) log/i w (y 



= X> • v(fc) lo s h (v) ~ M v (fc) log h\y {i) } 



+ 



R N xl 



n(de,d7/|y), 



where n(<i£, t^jy) denotes the conditional product measure n"=i 7r ^' ) (d^ e \dr]^ |y^). 
It follows that 



E p { V X • V log h] = J2 Pk ■ Ep| V« log /i| 2 
fc=i 

E p |V (fc) log/i| 2 -E p |E p {V (fc) log/i|y w } • V (fc) log/i 
+ E p , n { E [V( fc )y(yW,e W ) -V%(yW,^)] -V^g/A 



+ E « 

fc,^e[l,n],^fc 



Here E P) n denotes expectation with respect to the joint distribution £(Y,£,rj), 
defined by C(Y) = p and £(f , 77 |r) = n(df, drj\y). 



For k ^ £ we have 



E T 



V (fc) log%) -E p {V (fc) log/i|y w } 



V W log%) 



E T 



V W log%) 



-Ep EjjV^log/^} >0 



(3.19) 



To estimate the last line in (3.18), we introduce the notation 

U{y,t)= E W-vWy(yW,eW).vWlog%), y,£ € 

fc,^e[l,n],^fc 



We have 

E 

k,ee[i,n],e^k 



vW7(j|W,^)-vWy(yW,ijW) 



■V (fc) log/ i (y) = C/(y,0-^(y,r 7 ). 



To estimate |?7(y, £) — t/(y, t/)|, we carry out the followong calculation: 
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d_ 



E E Pi ■ & " H) ■ ViAv W >1 (£) + <t W ~ ■ diloghiy). 

k,ee[i,n],e^k iei k ,jeh 



Hence, by Assumption 3 (c.f. (2.5)), 
d 



dr 



U(y,r) + T(£-r))) 



< 



\ t=i jeh 



\ k=neh 



v D«-p)-p?-(e w -»/W) 5 



v E(^-p)-i v(fe)1 °g%)i 2 - 
\ k=i 



It follows that for all y, £ and 77 



E p*- 

k,ee[i,n],e^k 



v(%(^UW)-v(%(^,/)) 

U(y,t)-U(y,ri) 



• V (fc) log%) 



< 



v E(^-p)-l v(fe)1 °s^)l 2 - 
\ k=i 



Now the last line of (3.18) can be estimated as follows: 



E 



E [V<*V(y«\ - VWF^,/))] • V<*>log* 

fe,€e[l,n],£^fc 



< 



v E(^-P)-^- E P.n|^-^)| 
\j £=1 



E(^-^)- E pl v(fc)1 °g%)l 2 - 

\| fc=i (3.20) 



Our calculations are valid for any coupling of the conditional densities \y^) 
and Q( l \dqW\yW). Now we specify 7T^\d^ £ \drj^\y^) so as to achieve 

K* t {\v W -Z W \ 2 \v W } = W*U%\y<%Q<%\yW)) for any y«> . 
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By Assumptions 1 and 2, the Otto-Villani theorem can be applied to Q^'{-\y^')- 
Using also the logarithmic Sobolev inequality for Q^'{-\y^), we get 

Mto w -^ } l 2 1 y (e) } < y e -D(p (e H¥ e) ) II Qi(¥ e) )) 



<^-l{p W (¥ e) ) || Q {£) (¥ e) )) = ^•E p {|V^log/,| 2 | yW} 
for any yW. Substituting (3.21) into (3.20): 



(3.21) 



n 

<^(pfc-p)-E P |V«log/i| 2 . (3.22) 



fc=i 



Substituting (3.19) and (3.22) into (3.18): 

n n 

E P {V X -Vlogh}>Y,Pk- E P |V« log/i| 2 - J2( P k ~ P) ■ E P |V« \ogh\ 2 
fc=i fc=i 

= p- ^|V (fc) log/i| 2 = p-E p |Vlog/i| 2 . □ 



fe=i 
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4. Proof of the Auxiliary Theorem and the Approximation Lemma 

In the proof of the Auxiliary Theorem we use the weighted Gibbs sampler T with 
weights pk/R, R = J2kPki defined in Section 2: 

n 

^•X>-r fe , r k (z\y) = %<*>,*<*>) • gW(z(*)|yW). 



R 



k=l 



Recall that 



sup||i4(a;,0|| =1-*<1. 



Proposition 4. Under Assumptions 1-3, for fixed z,u E M we have 

J^PfW 2 (qM (.|z<*>), Q<*> {.\u^) \ <2-J^D (qM (.|z(*>) || Q<*> (.|«<*>)) 
fc=i ^ ' fc=i ^ ' 



<(1-*) 2 . J> 



U<*> -«<*>! 



(4.1) 



fc=i 



Proof. 

The first inequality follows from the Otto-Villani theorem for Then 
we use the logarithmic Sobolev inequality to continue (4.1) as follows: 

<£l.j/Q(*)(.| f <*)) || g(*)(.| ii (*)) > ) 

^ fc =i ^ fc i=i (4.2) 



To estimate the sum under the integral in (4.2), fix r) N , and consider the function 
F= (F 1 ,...,F N ) :R N ^R N defined by 



,(1) 



z (fc-i) v (k) z (k+i) 



F W {z) = J_. V (k)y(l 

V 7 ^ Vv^l y/Pk-l \/Pk y/Pk+1 



y {n) 



With the notation 

C (fc) = « W ' ^ (fc) = « (fe) -v^, fcG[l,n] 



the sum under the integral in (4.2) is just the squared increment of F between 
points C and 9: 



^_L. V<*V(zW,T7W)-V<*V(«W,»7W) = £ 
k=l Pk k=l 



2 n 



F (fc) (C) -F^ k \e) 



■ (4-3) 
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The Jacobian of F is 

^•M* (fc w* ) ) , y 

(It has zeros for i and j belonging to the same Ik-) Thus, by (2.4), 

n n 

EI F *(o - ^wi 2 < (i - • E px ■ \ z(k) - u(k) \ 2 - ( 4 - 4 ) 

Substituting (4.3) and (4.4) into (4.2) we get the desired result (4.1). □ 

We use Proposition 4 to show that the Gibbs sampler T is a contraction with 
respect to a weighted Wasserstein distance. 

Definition. Let r and s probability measures r and s on M. N . We define the 
weighted quadratic Wasserstein distance of r and s (with wights pk) by 

n 

Wl k} (r,s) = mfJ2Pk-^\ z(k) ~ U(k) \ 2 ' 
k=i 

where Z and U are random sequences s with laws r resp. s, and infimum is taken 
over all distributions 7r = C(Z, U) with marginals r and s. 

Proposition 5. 

If Assumptions 1-3 hold for q then 

%}(^ S r)<(i-^).% } (y). (4.5) 

Proof. 

Let Z = (Zi, Z2, • • • , Zn) and U = (U\, U2, . . .,Un) be random sequences in WL N , 
with C(Z) = r, C(U) = s, and let 7r = U) be that joining of r and s that 
achieves W{ Pfc } (r, s). Select a random index k £ [1, n] according to the distribution 
(pk/R), and define 

c(z'\z, u) = r K (-\z), c(u'\z, u) = r K (-\u). 

Then C(Z') = rT, and C(U') = sF. Further, define C(Z', U') as that coupling of rT 
and sT that achieves W 2 (Q^ K \-\Z^),Q^ K \-\U^)) for each value of the condition. 
Thereby we have defined C(Z', U'\Z, U), and by Proposition 4 we have 

W L} K sF ) < E ^ • (l - f + f • (1 - 5f) ■ E|Z<*> - | 2 
fc=i ^ ' 

/ _ 2 Pmin .6-(l-6/2)\ j^ p ^ nz{k) _ u{k)] 2 
^ R ' fc=i 



= 1 



Pmin ' & 



< (l _^). w ^ )(r , s) . □ 



In the sequel we shall use the 
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Notation. 

7(p(*)(.|y(«»)) || g( fc )(.|y( fc ))) 4E/(p( fc )(-|F^) || Qi(-\Y W )) 
(omitting the symbol of expectation). 

Proposition 6. 

Under Assumptions 1-3 we have 



W iPk} (p,q)< 
2R 

Pmin ' $ \ 



2R 



^ Pfc .EW2(p(*)(.|y(*)),Q(*)(.|y(*)) 



^1 . j(p«(.|y«) || Q(*)(.|y(*))^. 



Proof. 

The first inequality follows from the triangle inequality for W{ Pfe }(p, g) and Propo- 
sition 5, and the second one follows from the the Otto-Villani theorem and the loga- 
rithmic Sobolev inequality for Q^ k \-\y^) . □ 

Proposition 7. 

There exists a C = C(R, p m in, 8) > (R = J2 k pk and p m i n = min^ pk) such that 



n 

£j>(y(*) ||x( fc ))<^.J(p||g). 



(4.6) 



k=i 



Proof. 

Let 7f = C(Y,X) denote that joining of p = C(Y) and q = C(X) that achieves 
W {pk} (p, q). 

The convexity of the entropy functional implies the inequality 

n n / \ 

J2D(Y^ II XW) < J2^ D ( Yik) \ Yik) || Q ik) {-\X^)Y (4.7) 
fc=i fc=i ^ ' 

The right-hand-side of (4.7) can be written as a sum of three terms: 



J2^d(y^\y^ \\q^(-\x^)\ 

k=l ^ ' 



E 



+ ( 

" XIK " k=i 
= Si + S2 + Ss 



p(*)(y(*)|y(*))-Q(*)(y(*)|yW) 



(fc) f„( fc )lii( fc )' 



^Wy(*)|y(*) || g( fc )(.|y( fc )) j +^E 7rJ D(g«(-|y«) || 
fc=i ^ ; fc=i 

Q(*)(y(*)|y(*)) 



Q(k) ( y (fc)|£(fc)) 



(4.8) 
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By the logarithmic Sobolev inequality for (-{y^) we have 



Sl < i.£lj(V(*)|y(*) || g<*)(.|yW)). 



(4.9) 



Further, by Propositions 4, 



S 2 < ■ £ W • K,\Y«> - X<^ = <i-^ ■ W {n}M . (4.10) 



fc=l 



£3 can be written as 



s 3 = e m ]T 



k=i 



V(Y) - 7(I w ,y w ) - V(Y < fc >, C (fc) ) + 



(4.11) 



where // = £(Y,X,£) is defined by £(Y,X) = tt, £(f |Y, X) = flLi and 
£(y(fc)^(*)|y(fc)) i s an arbitrary joining of p( fc )(.|y( fc )) and Q( fc )(-|E^). 

We claim that for any quadruple of sequences (y N , r] N , x N , £ N ) the following 
inequality holds: 



E 

fe=i 



V(y(*), ^W) - V(x {k \^ k) ) - V(y {k \ f <*>) + V(x (fc) , f (fc) ) 



<(1-S) 



x ^2pk\y {k) - x (k) \ 
\ k=i 



\ k=i 



(4.12) 



Indeed, introducing the function 



X K i-> 



F(y,r,) = Y,V(y W ,V {k) ] 



k=i 



the left-hand-side of (4.12) can be rewritten az follows: 



E 

k=l 



V(y ik \r]W) -V(x {k \ V ^) -V(y {k \C (k) ) + V(x {k) ,C {k) ) 



= F(y, rj) - F(x, V ) - F(y, £) + F(x, £)• 



(4.13) 



To estimate the right-hand-side of (4.13) (with y,x,r),£ G M, N fixed), define 

G : [0, 1] x [0, 1] H> R, 

G(s,t) = F(x + s(y-x),S + t(7i-Z)) 

n 

= J2v{x (k ) + s (y<*> _ X W)^W + t{r] W _ £<*))). 



k=i 
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Then we have 



F(y,r)) - F(x, V ) - F{y,£) + F(x,£) 
= G(l, 1) - G(l, 0) - G(0, 1) + G(0, 0). 



(4.14) 



We have by (2.4) 
d 2 



d s d t 



■G(s,t) 



<(1-S) 



\ k=l 



Pk ■ \y^ — I 



X>-|t7<<>-^>| 2 . 



Putting together (4.13), (4.14) and (4.15) yields (4.12). 
Applying (4.12) for i] = y: 



E 

fe=i 



V(y) - V(x {k \y^) - V{y {k \ £<*>) + K(z<*>, £<*>) 



<{l-6) 



\ k=i 



(4.15) 



(4.16) 



Substituting (4.16) into (4.11), and using Jensen's inequality, we get 



5 a < 



fe=i 



E^- E l y(fc) -e (fc) r 
\ fc=i 



= W {pk} {p,q) 



\ fc=i 



(4.17) 



To estimate the second factor, we select for C{Y^ k \ ^ \Y^) that joining of 
the mare; inals that achieves W 2 (p^(-\Y (*)), QW(-\YW)) for every value of the 
conditions. Then the Otto-Villani theorem and the logarithmic Sobolev inequality 
for Q^(-\y^) imply the following bound for S3: 



S 3 <W {pk} (p,q) 



n , 

E- ■l(p^(-\Y( k ))\\Q( k )(-\Y( k )) 



(4.18) 
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Putting together (4.9), (4.10) and (4.18) 
5i + S2 + S3 
1 



< 



W {pk} (p,q) + 



\k=i Pk ^ 



(4.7), together with (4.8) and (4.19), completes the proof of Proposition 7. 



(4.19) 

□ 



Proof of the Auxiliary Theorem. 

The proof goes by induction on n. It is clear that for any k and G 
Assumptions 1-3 formulated before Theorem 1 do hold for n = 1, N = \I k \ and the 
distribution Q^ k \-\y^) . Assume that we have proved the Auxiliary Theorem for 
n — 1 in place of n. 

By a well known identity for relative entropy, we have 
D(p\\q) = D(Y\\X) 

= ~Y j D(Y^ || X™) + ^f2 D (^ (k) \ Yik) II q {k) {-\Y^)). 

n k=i n k=i V ' ' (4.20) 

Assume the Auxiliary Theorem for n — 1. By the induction hypothesis, 
D (y(k) lY (k) ||g(*)(.|y(*))^ <_L.^/(yW|yW || QW(.|yW)^ for all jfc. 



Thus 



I^D(y(fc)|y(*) || ^)(.|y( fc ))) 
re fc=i 

^(l-Vn)~E J ( yW l^ W II ^M^)) 



(l-l/n)~-/(p||«). 



(4.21) 



Substituting (4.6) (Proposition 7) and (4.21) into (4.20) completes the proof of 
the Auxiliary Theorem. □ 

Proof of the Approximation Lemma. 

First we keep q fixed, and construct a density g G C°° with compact support, and 
such that, with the notations 



r = g-q, r 



(fc) 



R( fe ) 
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we have 

D(r\\q) is arbitrarily close to D(p\\q), (4.22) 

and 

D(f {k) \\q {k) ) is arbitrarily close to D(p {k) \ \q^ k) ) for all ke[l,n]. (4.23) 

Denote by B m the closed ball in R N around the origin and with radius m. Let 
4> m : R N !->■ [0, 1] be a C°° function satisfying 

(pm(x) = 1 for x e B rn , 4> m (x) = for x B m+1 . 

Set s 

g m (x) = h(x) ■ (f> m (x), and r m (x) = g m (x) ■ q(x), 

where a m = J RN h(x) ■ <j) m (x)q(dx). 
We have 

D(r m \\q) = — I ( h(x)<f> m (x) J • log (/i(x)0 m (x) J - log a m , 
otm Jrn \ J \ J 

and lim^i a m = 1. Since h(x) ■ 4> m (x) — > h(x) everywhere, with \ {h{x)4> m (x)^ ■ 
\og{h{x)4> m {x)^ | increasing, and using also the inequality ulog-u > — 1/e, it fol- 
lows that 

lim / (^(a;)0 m (x) ) -log(/i(a;)0 TO (x) = D(p||g). 



Putting g = g m and r = r m , for large enough m we achieve (4.22). It can be proved 
similarly that (4.23) can be achieved as well. 

Again, it is easily seen that 

limD^(l-e) ■ r + e ■ q) \\ qj = D(r\\q), 

and 

lim d((1 - e) • r (fc) (-|F«) + e ■ q {k \-\Y {k) ) \\ q {k) (-\Y^) 
= £>( r (*)(.|y(*)) \\q( k \.\Y^). 

Thus, for q fixed, h can be replaced by / = (1 — e)g + e. 
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Now we can assume that h is of the form claimed in the Approximation lemma. 
We keep the notation p = h ■ q with the newly defined h, and keep h fixed. 

Now we approximate q(x) by an increasing sequence q m (x) G and set 

q m = and (*<*>)= / q m (x)dx^ . 

J q m (x)dx J R (k) 

Then define p m (x) = h(x) ■ q m (x). Since h is smooth and bounded from below and 
above, it is easily seen that 

D{pm\\q m )= / h(x) log h(x)q m (dx) -> D(p\\q), 
Jr n 

and 

D @m II Qm)) = f h^ k \x {k) )-\ogU k \x {k) )q^\dx^) ^D(p^\\q^). 
This completes the proof of the Approximation Lemma. 
Acknowledgment 

The author is grateful for the patient support and help by P. E. Frenkel. 



References 

[I] L. Gross, Logarithmic Sobolev inequalities, Amer. J. Math. 97 (1975), 1061-1083. 
[2] E. Nelson, The free Markov field, J. Funct. Anal. 12 (1973), 211-277. 

[3] F. Otto, C. Villani, Generalization of an inequality by Talagrand and links with the logarithmic 
Sobolev inequality, J. Funct. Anal. 173 (2000), 361-400. 

[4] Th. Bodineau, B. Helffer, On Log-Sobolev inequalities for unbounded spin systems, Funct. 
Anal. 166, (1999), 168-178. 

[5] B. Helffer, Remarks on decay of correlation and Witten-Laplacians III. Application to loga- 
rithmic Sobolev inequalities, Ann. Inst. H. Poincare, 35, (1999), 483-508. 

[6] D. Bakry, M. Emery, Diffusions hypercontractives, Seminaire de Probabilites, XIX. Lecture 
Notes in Math. 1123, Springer, 1985, pp. 177-206. 

[7] R. Holley, D. Stroock, Logarithmic Sobolev inequalities and stochastic Ising models, J. Statist. 
Phys. 16 (1987), 1159-1191. 

[8] B. Zegarlinski, Dobrushin's uniqueness theorem and logarithmic Sobolev inequalities, J. Funct. 
Anal. 105 (1992), 77-111. 

[9] G. Royer, Une Initiation aux Inegalites de Sobolev Logarithmiques, Soc. Math, de France, 1999. 

[10] F. Otto, M. Reznikoff, A new criterion for the logarithmic Sobolev inequality and two appli- 
cations, J. Funct. Anal. 243, (2011), 121-157. 

[II] M. Talagrand, Transportation cost for Gaussian and other product measures, Geometric and 
Functional Analysis, 6 (1996), 587-600. 

[12] S. Bobkov, I. Gentil, M. Ledoux, Hypercontractivity of Hamilton- Jacobi equations, J. Math. 

Pures Appl. 80 (2001), 669-696. 
[13] K. Marton, Measure concentration for Euclidean distance in the case of dependent random 

variables,, Ann. Prob. 32 (2004), 2526-2544. 



LOGARITHMIC SOBOLEV INEQUALITIES 



29 



[14] K. Marton, Correction to the paper "Measure concentration for Euclidean distance in the 
case of dependent random variables", Annals of Probability . 38 (2010), 438-442. 

[15] U. Grenander, G. Szego, Toeplitz forms and their applications, University of California Press, 
Berkley, 1958. 

[16] F. Riesz, B. Szokefalvi-Nagy, Functional analysis, Ungar, New York, 1955. 



H-1364 POB 127, Budapest, tel. 36 (1) 4838300, Hungary 
E-mail address: marton@renyi.hu 



